Search for: All records

Graphical representations are ubiquitous in the learning and teaching of science, technology, engineering, and mathematics (STEM). However, these materials are often not accessible to the over 547,000 students in the United States with blindness and significant visual impairment, creating barriers to pursuing STEM educational and career pathways. Furthermore, even when such materials are made available to visually impaired students, access is likely through literalized modes (e.g., braille, verbal description), which is problematic as these approaches (1) do not directly convey spatial information and (2) are different from the graphic-based materials used by students without visual impairment. The purpose of this study was to design and evaluate a universally accessible system for communicating graphical representations in STEM classes. By combining a multisensory vibro-audio interface and an app running on consumer mobile hardware, the system is meant to work equally well for all students, irrespective of their visual status. We report the design of the experimental system and the results of an experiment where we compared learning performance with the system to traditional (visual or tactile) diagrams for sighted participants (n = 20) and visually impaired participants (n =9) respectively. While the experimental multimodal diagrammatic system (MDS) did result in significant learning gains for both groups of participants, the results also revealed no statistically significant differences in the capacity for learning from graphical information across both comparison groups. Likewise, there were no statistically significant differences in the capacity for learning from graphical information between the stimuli presented through the experimental system and the traditional (visual or tactile) diagram control conditions, across either participant group. These findings suggest that both groups were able to learn graphical information from the experimental system as well as traditional diagram presentation materials. This learning modality was supported without the need for conversion of the diagrams to make them accessible for participants who required tactile materials. The system also provided additional multisensory information for sighted participants to interpret and answer questions about the diagrams. Findings are interpreted in terms of new universal design principles for producing multisensory graphical representations that would be accessible to all learners. 

Ultrasonic haptic (UH) feedback employs mid-air ultrasound waves detectable by the palm of the hand. This interface demonstrates a novel opportunity to utilize non-visual input and output (I/O) functionalities in interactive applications, such as vehicle controls that allow the user to keep their eyes on the road. However, more work is needed to evaluate the useability of such an interface. In this study, 16 blindfolded participants completed tasks involving finding and counting UH buttons, associating buttons with audio cues, learning spatial arrangements, and determining button states. Results showed that users were generally successful with 2–4 arranged buttons and could associate them with audio cues with an average accuracy of 77.1%. Participants were also able to comprehend button spatial arrangements with 77.8% accuracy and engage in reconstruction tasks to prove user understanding. These results signify the capability of UH feedback to have real-world I/O functionality and serve to guide future exploration in this area. 

Mid-air ultrasonic feedback is a new form of haptic stimulation supporting mid-air, touch-free user interfaces. Functional implementation of ultrasonic haptic (UH) interfaces depend upon the ability to accurately distinguish between the intensity, shape, orientation, and movement of a signal. This user study (N = 15) investigates the ability to non-visually perceive two ultrasonic lines with varying lengths (3, 5, and 7 cm) and orientations (vertical and horizontal) using the palm of the hand. Key results showed that: (1) the orientation of the lines had no effect on a user’s accuracy when determining their relative lengths, (2) line length distinction significantly improved when the length difference was at least 4 cm, and (3) a clear learning curve was evident when evaluating a new user’s ability to perceive ultrasonic signals. The capabilities of UH technology identified and discussed within this study will help engineer user-friendly and functional mid-air haptic interfaces for future applications.